Echoprint user authentication

ABSTRACT

A device and method providing audio-based, continuous authentication of a user to provide access to a processing device. A user echoprint model is created while a user interacts with an authentication interface for another authentication method of the device, transparently to the user. The sound waves are user-specific and inaudible. The user-specific sounds are reflected by the user and used to construct an echoprint which can be compared by a previously constructed user-specific, echoprint model. If the result is positive, access to the processing device may be allowed. Access to the processing device is based on the audio authentication or the alternative authentication is based on the result of audio authentication and updates a device state indicating whether the display is on or off and whether the device is locked or unlocked.

BACKGROUND

Various authentication mechanisms are currently utilized to preventunauthorized access to processing devices such as personal computers andmobile devices. Password-based logins have given way to biometric formsof authentication such as fingerprint readers and facial recognition.Current biometric forms of authentication generally require some specialform of biometric sensors be coupled to the processing device.

BRIEF SUMMARY

One general aspect of the present disclosure includes a processingdevice having audio-based authentication, including: a display; amicrophone; a speaker; an audio signal generator coupled to the speakeradapted to output a user-specific inaudible sound to the speaker; asignal processor coupled to the microphone adapted to isolatereflections of the user-specific inaudible sound signal and create anechoprint based on the reflections; and a processor configured tocompare the test echoprint with a user-specific echoprint modelassociated with the user; perform an alternative authenticationoperation to authenticate the user based on the comparison; and update adevice state according to the comparison or the alternativeauthentication operation performed, with the device state comprisingwhether the display is on or off and whether the device is locked orunlocked

Optionally, in any of the preceding aspects the device includes amachine learning processor is adapted to create the user-specificechoprint during an alternative authentication set-up procedure.Optionally, in any of the preceding aspects the echoprint identifiesfeatures associated with the user, such features including at leastbiometric features or behavioral features. Optionally, in any of thepreceding aspects the biometric features include at least physicalattributes or cardiac attributes. Optionally, in any of the precedingaspects the machine learning processor is adapted to build theuser-specific echoprint model during a period less than or equal tobeginning when a user enables authentication through when a user selectsto enable audio-based authentication. Optionally, in any of thepreceding aspects the signal processor is adapted to build theuser-specific echoprint during a period between entry of a passwordentry user interface through a user-interface prompt suggesting the userenable audio-based authentication. Optionally, in any of the precedingaspects, the device further includes a non-volatile storage deviceadapted to store one or more echoprints generated by the signalprocessor as a training data set and the user-specific echoprint model.Optionally, in any of the preceding aspects the machine learningprocessor is adapted to receive user feedback on authenticationrecognition and update the user-specific echoprint model based on thefeedback. Optionally, in any of the preceding aspects the audio signalgenerator and signal processor are adapted to operate while the displayis on and the device is locked, and are adapted to remain in standbywhen the display is off and the device is locked.

Another general aspect includes a processor-implemented method ofauthenticating user access to a processing device having a display,including: generating a user-specific echoprint model using a machinelearning process by outputting an inaudible sound signal at auser-specific frequency from a speaker associated with the processingdevice, and receiving reflections of the inaudible sound signal at amicrophone associated with the processing device while the userinteracts with an alternative authentication user interface;authenticate the user access when the display is on by generating a testechoprint by outputting an inaudible test sound signal at auser-specific frequency from the speaker, and receiving reflections ofthe inaudible test sound signal at the microphone; comparing the testechoprint with the user-specific echoprint model to determine whether toallow the user to access the processing device; performing analternative authentication method based on the comparison; and allowingor denying access to the processing device based on the testing byupdating a device state according to the comparing or the alternativeauthentication operation performed, the device state comprising whetherthe display is on or off and whether the device is locked or unlocked.

Optionally, in any of the preceding aspects the generating a testechoprint, the receiving reflections of the inaudible test sound signal,and the testing occur while the display is on and the device is locked,and do not occur when the display is off or the device is locked.Optionally, in any of the preceding aspects the outputting and receivingfor generating the user-specific echoprint model are repeated aplurality of times to create a training dataset for a machine learningprocess. Optionally, in any of the preceding aspects the testing outputsa different user-specific inaudible test sound signal upon anauthentication failure, and the testing isolates reflections of thedifferent user-specific inaudible test sound signal and creates a newtest echoprint based on the reflections of the different user-specificinaudible test sound signal. Optionally, in any of the preceding aspectsthe generating comprises identifying features associated with the user,such features including at least biometric features or behavioralfeatures. Implementations of the described techniques may includehardware, a method or process, or computer software on acomputer-accessible medium.

Another aspect includes a non-transitory computer-readable mediumstoring computer instructions, that when executed by one or moreprocessors, cause the one or more processors to perform the steps of:generate an echo-print model for a user by: generating an echoprintmodel for a user by: outputting an inaudible sound signal at auser-specific frequency from a speaker associated with the processor;receiving reflections of the inaudible sound signal; generating anecho-print based on the reflections and associating the echo-print witha user; repeating the outputting, receiving and generating to createtraining data. The non-transitory computer-readable medium also includesauthenticating a user to provide access to a processing device having adisplay, by, when the display device is on and the device is locked:outputting a test inaudible sound signal at the user-specific frequencyfrom a speaker associated with the processing device; receivingreflections of the test inaudible sound signal; generating at least onetest echo-print based on the reflections of the test inaudible soundsignal; comparing the echoprint by a user-specific echoprint model todetermine whether to allow the user to access the processing device;performing an alternative authentication method based on the comparing;and allow or denying access to the processing device based on thetesting and the alternative authentication method by updating a devicestate according to the comparing or the alternative authenticationoperation performed, the device state comprising whether the display ison or off and whether the device is locked or unlocked.

Optionally, in any of the preceding aspects, the non-transitorycomputer-readable medium storing computer instructions includesinstructions where outputting a user-specific inaudible sound includesoutputting a different test user-specific inaudible sound and the signalprocessor adapted to isolate reflections of the different testuser-specific inaudible sound and create an echoprint based on thereflections. Optionally, in any of the preceding aspects, thenon-transitory computer-readable medium storing computer instructionsincludes instructions where each outputting sends the user-specificinaudible signal. Optionally, in any of the preceding aspects thenon-transitory computer-readable medium storing computer instructionsincludes instructions which cause the one or more processors to updatethe user-specific echoprint model using a machine learning process basedon user feedback of whether the testing was accurate. Optionally, in anyof the preceding aspects, the non-transitory computer-readable mediumstoring computer instructions includes instructions where the echoprintidentifies features associated with the user, such features including atleast biometric features or behavioral features. Optionally, in any ofthe preceding aspects, the non-transitory computer-readable mediumstoring computer instructions includes instructions wherein thebiometric features include at least physical attributes, or cardiacattributes.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the present disclosure are illustrated by way of example andare not limited by the accompanying figures for which like referencesindicate like elements.

FIG. 1 illustrates a first embodiment of a processing deviceillustrating an overview of the present technology using audio signalsand echoprints to authenticate user access to a processing device.

FIG. 2A is a flowchart illustrating a selection process for determininga type of authentication to use with a processing device.

FIGS. 2B-2D are depictions of user interfaces illustrating a set-upprocess for a processing device.

FIG. 3 is a flowchart illustrating a process of acquiring an audioreflection based echo signature.

FIG. 4. is a flowchart illustrating building a user model using machinelearning.

FIG. 5 is a flowchart illustrating testing and updating of a userechoprint model.

FIG. 6A illustrates an authentication process in accordance with thetechnology.

FIG. 6B illustrates one embodiment of a testing processor suitable forimplementing at least a portion of the process of FIG. 6A.

FIG. 6C illustrates one alternative for the processing of reflectedsignals in step 625 of FIG. 6A.

FIG. 7 illustrates a status of an audio authentication method and analternative authentication method based on the display state and thelocked vs. unlocked state of the processing device.

FIG. 8 illustrates a mobile processing device in accordance with thepresent technology.

DETAILED DESCRIPTION

Technology is presented which enables an audio-based, continuousauthentication of a user to provide access to a processing device. Theaudio-based authentication uses sound waves generated and received bystandard components usually found in processing devices, enablingincorporation of the technology into existing devices as well as newdevices. Speakers associated with or incorporated into a processingdevice emit a user-specific inaudible sound. The user-specific soundsare reflected by the user and used to construct an “echo” print (anaudio signature incorporating various biometric and behavioral featuresassociated with the user) which is tested by a well-trained echoprintmodel. If the test result is positive, access to the processing devicemay be allowed.

In one aspect, training of the echo print model may occur while the userinteracts with a user interface to enable an alternative form ofauthentication, such as password-based authentication. In this aspect,the device may emit user-specific inaudible sound while the user iscreating a password and train the echoprint model, then prompt the userto enable an audio authentication system, also referred to herein as“EchoID.”

Access to any processing device, or resource on a processing device, maybe restricted until an authorized user authenticates. The audioauthentication system may be combined with any suitable alternativeauthentication system. In some embodiments, the alternativeauthentication system may include a system for detecting biometricfeatures or attributes of a user. For example, the processing device mayinclude a biometric fingerprint system operative to detect andauthenticate a user based on features of or under a user's skin.

The processing device 100 may provide any suitable combination ofauthentication systems, including for example biometric authenticationsystems and/or pattern-based authentication systems (such as passcodeauthentication).

FIG. 1 illustrates a processing device 100 suitable for implementingaudio-based authentication interacting with a user 10. Processing device100 may include a microphone 102, a speaker 104 and other componentsconnected by a bus 106. As illustrated in FIG. 1, the components of theprocessing device 100 include signal processor 110, a machine learningprocessor 120, an echo signature storage device 130, a testing processor140, and a user-specific audio generator 150. Other device hardware 160may be provided in the processing device, including a user-interfacegenerator 170 creating and outputting various user interfaces to adisplay, as described herein. Specific examples of processing device 100are illustrated in FIG. 8. In processing device 100, an audio-basedauthentication of a user 10 may be performed by first causing theuser-specific audio generator 150 to generate a user-specific inaudibleaudio signal 50, which is output from the speaker 104. The user-specificinaudible sound may be in the audible frequency range, but at a volumewhich is too low for the human ear, or in the infrasound or ultrasoundfrequency range. Reflections 55 of the user-specific inaudible audio 50are received by and detected by the microphone 102 and provided tosignal processor 110. Signal processor 110 processes the reflected audio55 (or “echos”) and creates an echoprint. The echoprint may comprise anaudio signature in which a combination of various biometric andbehavioral features derived from the reflections 55 are identified. Whenis tested by a user-specific echoprint model which identifies thosefeatures of the signature which can be matched to an individual user,reflections 55 can be used to identify and authenticate user 10.

The user-specific audio signal may be designed to comprise a signalwhich is emitted from the speaker which has a high statisticalprobability of being unique to that user's processing device relative toanother processing device which may be nearby. This may be achieved bycreating the user-specific audio signal with randomized elements in thesignal, including elements of time, frequency and/or amplitude, or anyother variation of an audio signal which may be emitted by the speaker104.

An echoprint storage device 130 is a nonvolatile storage device whichcan be accessed by the processors in order to perform the functionsdescribed herein. The testing processor 140 tests the newly receivedreflections during an authentication process to an echoprint user modelto perform an authentication. The echoprint user model is created by themachine learning processor 120, as described herein. The machinelearning processor 120 can access feedback from authentication events toimprove the echoprint user model.

Each of the processors illustrated in FIG. 1 may be implemented asseparate processors dedicated to providing the functions describedherein implemented by a dedicated hardware processor, a programmableprocessor including a combination of hardware and instructions operableto cause the processor to perform the described functions, or as ageneral purpose single processor executing instructions operable toinstruct a single processor to perform the described functions. Thus, asillustrated in FIG. 8, each of the “processors” illustrated in FIG. 1may be implemented as a process created by executing the instructionsoperable to instruct one or more processors to perform the functionsassociated with the processor of FIG. 1.

In order to implement security on a processing device, generally a useris provided with an opportunity to select the type of authentication theuser wishes to use with the processing device. Various forms ofalternative authentication may be utilized with audio-basedauthentication described herein. The technology provides an easilyusable form of authentication using hardware already incorporated intomost mobile devices, such as a speaker and microphone. The technology isalso unobtrusive to the user, using an inaudible sound signal to performauthentication.

FIG. 2A illustrates a method allowing a user to select the type ofsecurity authentication they wish to utilize in a processing device.Initially, at 200, a security selection operation is performed. FIGS.2B-2D illustrate exemplary user-interfaces on a processing device(displayed, for example, on a display 852 of a mobile device 800 of FIG.8). The interfaces may be created by the user interface generator 170.The security selection operation can be performed by a user accessing aprocessing device user interface settings screen to change a securitiesselection, or may be automatically generated when a user accesses aprocessing device for the first time. For example, the process of FIG.2A may be performed during an initial device set-up process, or at somepoint after a user has been operating a device and decides that theywish to switch authentication modes.

At 210, a user may be prompted for a selection of security preferences.The prompt may take any form of display in a user interface provided ona display device associated with the processing device. Device 100 a maybe a processing device such as device 800 illustrated in FIG. 8. In oneembodiment, a SETTINGS prompt may comprise a check box which the userselects to “enable authentication”, and an option may be provided toallow the user to set a numeric or alphanumeric password. On a touchscreen interface such as that illustrated in FIG. 8, a user may touch acheck box 205. As illustrated in FIG. 8, a display may be integrated ina housing with processing device, or provided separate from a housing ofthe processing device. Once prompted at 210, the user may select toenable security at 220 by, for example, touching check box 205. If theuser does not select to enable security at 220, then no security may beutilized with the processing device at 230. If the user selects toenable security at 220, and the user may select to enable a passcode at240. In the user interface of FIG. 2B, this may be performed byselecting arrow 215 which provides a user interface such as that shownin FIG. 2C.

In one embodiment, at 250, while the user enters a passcode in the UI ofFIG. 2C, an echoprint model for the user is trained. Training can occurduring a period beginning with the selection of the enableauthentication check mark 205, or beginning when the user selects thearrow 215. The period may continue while the user keys in the passwordusing a keyboard 235 through to the point that a prompt (illustrated inFIG. 2D) is shown to the user (or any sub-period of the foregoingperiod).

Once the user completes entering the passcode at 250, a prompt is issuedat 260. The prompt is illustrated at FIG. 2D. As shown therein, the userinterface may indicate a “password set” and contain a prompt 245encouraging the user to enable EchoID and a check interface 255 allowingthe user to enable selection of EchoID in addition to the passcodeauthentication.

Audio-based authentication or EchoID as discussed herein is based onsampling inaudible sound reflections of an emitted unique anduser-specific signal. FIG. 3 illustrates one method for outputting andreceiving reflections to create an echoprint for use in the audio-basedauthentication herein.

Initially, echoprint audio authentication is enabled at 300. Asdiscussed above, the user may have various options to enable varioustypes of authentication within the processing device.

At 320, a user-specific inaudible sound is emitted over a particularperiod of time T. The emitted sound at 320 is unique to each device. Theuniqueness of the sound signal allows the technology to isolatereflected sounds in creating the echoprint. The period of time T may bealternatively referred to as the sample period and may range fromnano-seconds to multiple seconds. At 330, the processing device receivesreflected user-specific inaudible sounds for a time T. The receipt timemay be time shifted with respect to the initial emission time of thesound signal at 320. The reflections or echoes will, after processing at325, show characteristics of a user which illustrate unique physicalcharacteristics, including cardiac motion, and biometric (human shape)data, and human motion activity. At 340 the method builds an echoprintbased on the received reflected sound and identifies data samples whichare unique to a user. At 350, the echoprint is output to a machinelearning algorithm or stored in a training data set which mayimplemented by the learning processor or stored in a training data setfor later use by the learning processor.

At 360, a determination is made as to whether additional samples shouldbe acquired. If an additional sample is to be acquired, then the methodreturns to step 320 and steps 330, 340, 350 and 360 are repeated.Generally, multiple samples are acquired to statistically create theechoprint model. To perform an authentication, one or many samples overmultiple sampling periods may be acquired before testing the samples tothe echoprint model for a user.

If no additional samples are to be acquired, the method ends at 370.After a sufficient number of samples has been acquired, a user-specificechoprint model may be created.

FIG. 4 illustrates a method of building a user-specific model for theechoprint characteristics of an individual user. At 410, echoprintsamples output by step 350 are received by a learning process orprocessor. The echoprint samples are associated desired output resultsfor both echoprints identifying a user and echoprints which do notidentify a user into one or more test data sets. At 420, a userechoprint model is calculated which characterizes those featuresuniquely associated with a user. The model may be created by machinelearning using any of a number of known machine learning techniques.

Any of multiple machine learning techniques can be utilized in thepresent technology. Each authorized user for whom the training data isavailable will have a model developed specifically for the user.

At 450, the model is stored and may thereafter be subjected to use intesting new samples to authenticate a user and may be updated by thelearning processor using a method in accordance with FIG. 5

Creation of an echoprint and an echoprint model may take many forms. Inone embodiment, an echoprint is a set of data comprising an N-dimensionvector. Each dimension may represent a feature or an element of afeature. Building an echoprint may comprise mapping the sound signal tothis set of data.

Once a user-specific model has been created, the model may be updatedbased on user feedback relative to the accuracy of the authenticationthe audio-based authentication makes.

FIG. 5 illustrates a method for testing and updating a user-specificechoprint model. At 500, the testing and updating process is enabled inthe machine learning processor or as a process in a general-purposeprocessor. During the testing process, at 510, a user-specific inaudiblesound signal is emitted over time T. At 515, reflections of the emitteduser-specific inaudible sound emitted at 510 is received. Signalprocessing occurs at 517 and an echoprint is built for the reflectedsignal at 518. Signal processing is discussed with respect to FIG. 6B.At 520, the echoprint is tested by the current version of theuser-specific echoprint model. Based on the test, an authenticationdetermination is made at 525. If the user is authenticated, or if theuser is not authenticated, a prompt to the user is made at 530 todetermine whether the authentication which was decided at 525 wascorrect. If the authentication was correct or incorrect, at 540, thedecision feedback 550 is provided to the learning processor. Thelearning processor receives the decision feedback at 560 and the corrector incorrect decision feedback is used to update a current version ofthe user-specific echoprint model at 570.

FIG. 6A illustrates a method which may be performed by the testingprocessor to authenticate a user for access to a processing device. Asfurther illustrated in FIG. 7, the testing method may be dependent on adevice state which reflects whether the display is on or off and whetherthe device is locked or unlocked. At 600, a device may be locked statewhich would require authentication for a user to access the device. Thedevice may also be unlocked but may be set to periodically checkauthentication even when unlocked, hence starting the process flow ofFIG. 6A. A user may engage with the phone to access the processingdevice thought a number of means which will activate the device displayat 605. For example, the user may press a physical button on theprocessing device, the user may lift the device, or the user mayinitiate a key sequence on a keyboard of the processing device. Steps610-640 provide an audio authentication process using a test echoprint.At 610, a user-specific inaudible sound is emitted for a samplingperiod. At 620, the user-specific inaudible sound reflections arereceived. At 625, the reflected signals are processed to extract andisolate characteristics of the reflected signal indicating specific userbiometric and behavioral characteristics. A method for processing thesignals is illustrated in FIG. 6B. At 630, a test echoprint is built. At635, the echoprint is tested by comparing the echoprint to the currentversion of the unique user model. At 635, the process compares the testechoprint with a user-specific echoprint model associated with the user.At 640, a determination is made based on the testing result. If so, theuser is authenticated at 640, and at 650, device access is provided byupdating the device state to allow device access such that the device iseither unlocked or remains unlocked at 650. If the test result isnegative, at 640, then the user is not authenticated by audio-basedauthentication and at 660, an alternative form of authentication (ifenabled) is provided to the user. At 650, the alternative authenticationoperation is performed to authenticate the user based on the comparisonat 635. If the user passes the alternative form of authentication at670, then access is provided at 650 by updating the device state. If theuser does not pass the alternative authentication at 670, then deviceaccess is blocked at 680 (i.e the device is locked) by updating thedevice state.

Optionally, steps 610-640 may be repeated one or more times with thesame or a different user-specific sound to attempt to authenticate theuser. These steps may be repeated after a failed attempt to authenticatea user.

Steps 635-680 may be performed by the testing processor 140 and oneembodiment of a testing processor 140 suitable for performing steps635-680 is illustrated in FIG. 6B. This testing processor 140 includescode operable to instruct the processor or processing circuitryspecifically implemented to compare the test echoprint with auser-specific echoprint model associated with the user at 641. Thistesting processor 140 includes code operable to instruct the processoror processing circuitry specifically implemented to perform analternative authentication operation to authenticate the user based onthe comparison at 643. This testing processor 140 includes code operableto instruct the processor or processing circuitry specificallyimplemented to update a device state according to the comparison or thealternative authentication operation performed, wherein the device stateindicates whether the display is on or off and whether the device islocked or unlocked at 647.

FIG. 6C illustrates a method of processing a signal in accordance withstep 630. At 628, ambient sound inputs are removed. In one embodiment,ambient sound may be removed. In one embodiment, ambient sounds may beremoved by filtering for reflections at the user-specific sound signal.

Feature identification may occur at 632 and involves creating a set ofdata to train the machine learning model at 420. Each dimension of theset of data corresponds to a feature. An echoprint is recorded as a setof data at 636.

A processing device may not require a user to authenticate each time theuser unlocks or operates the electronic device. In some embodiments, thecomputing device may allow a user to authenticate after a particularamount of time. The processing device may retain the user'sauthentication for a particular amount of time after having received theuser's last instruction or having entered a stand-by mode. The amount oftime the computing device retains authentication information may be setby the device or by the user.

FIG. 7 illustrates the on and standby states of two types ofauthentication which may be in simultaneous use on a processing device.Audio-based authentication such as that described above, may besimultaneously used with an alternative authentication method (such asfingerprint recognition or a passcode) to provide a user moreconvenience and ease in device operation. Various device status states710, 720, 730, and 740 b are shown. These four device states reflectwhether the device is locked or unlocked, and whether the device displayis on or off. The four possible device states are: device locked anddisplay off 710, device locked and display on 720, device unlocked anddisplay off at 730 and device unlocked and display on at 740 b. For eachdevice state, the status of each authentication mechanism is shown. Twodashed lines illustrate how device state and authentication methodstates change based on successful or unsuccessful authentication.

Typical processing devices include both power management functions andsecurity locking functions. Power management functions include optionssettable by the user to conserve device power by, for example, dimmingor turning off a display after a set time period of inactivity.Similarly, security settings allow a user to set both whether a deviceshould be locked and how long after inactivity an unlocked device shouldre-lock itself. It is possible that power settings may turn off a devicedisplay before the security settings re-lock a device, resulting in astate (730) where a device can be unlocked but the display is off. Ifthe display is turned on at state 730 (e.g. in response to detecting adisplay screen touch or a user command via a device button), the devicemay optionally utilize the audio authentication to re-verify(transparently to the user) that an authenticated user is accessing thedevice. In state 740 b, one form of authentication has successfullycompleted (as explained below) so no additional authentication isneeded.

At 710, the device is locked, and the display is off. This state mayoccur when a device is unused (for example, when the device is at restand power management functions in the device turn the display off). Inthis device state, the alternative authentication method is in standbyand audio-based authentication is off at 715. The state of “standby”means that the authentication method is not functioning but has beenactivated in the device setting by a user. As noted above, a user mayhave a choice not to activate any authentication or device locking atall, or may choose not to enable audio authentication and use some otherform of authentication. The flowchart of FIG. 7 illustrates that bothauthentication methods are enabled, but not necessarily active or in use(i.e. ON).

When the device is locked and the display is off (at 710), bothauthentication methods are in STANDBY mode. In STANDBY mode, the audioauthentication method is not outputting an inaudible signal and each ofthe processors may be powered down or off, in order to provide powerefficiency.

At 720, the device state is that the device is locked and the display ison. A device display may be activated when a user interacts with thedevice through a physical interaction mechanism such as a physicalbutton, or by moving the device which is detected by one or more motionsensors in the device. Once the display is on, the device state is suchthat it is assumed the user will likely need to access the lockeddevice. Hence, the audio-based authentication is enabled or turned on at725 and begins attempting to authenticate the user, while thealternative authentication method remains in standby at 725. This allowsunlocking of the device using the audio-based authentication method, ifsuccessful, prior to any indication to the user that the alternativeform of authentication may be needed. If authentication is successfulusing the audio-based method at 725, then the device is unlocked and thedisplay remains on at 740 b. If authentication is unsuccessful using theaudio-based authentication method at 725, then at 735, the audio-basedauthentication method remains on (and continues attempting toauthenticate the user) and the alternative authentication method isturned on. At 735, the user may be prompted with a message on thedisplay indicating that attempts to authenticate the user usingaudio-based authentication have failed and prompting the user to use thealternative form of authentication.

If authentication is unsuccessful under both methods, then the deviceremains in the locked state at 720 with the display on. If deviceauthentication is successful under either method 735, then the device isunlocked and the display is turned on at 740 b.

Once the device is unlocked and the display on at 740 b, bothauthentication methods may enter the standby mode at 715.

State 730 illustrates a state where the device is unlocked and thedisplay is off. As noted above, this may occur when the device powersettings cause the display to turn off after a shorter period of timethan the device security settings would cause the device to re-lockaccess to the device. The state of the device will change from that at730 to a state where the device is unlocked but the display is on at 740a with detection of user interaction with the device such as thatdescribed above. In this state, at 745, the audio-based authenticationmay be active to re-lock the device should the authentication methoddetect that the user interaction is not by an authenticated user. Forexample, if an authorized user unlocks the device, but then leaves thedevice and an unauthorized user picks up and attempts to access thedevice, the audio-based authentication can re-lock the device (return tostate 720). If the audio-based authentication errs at 745 (and anauthorized user is indeed interacting with the device), the methodfollows the flow from 720-735 allowing the user to authorize access inthe manner described above. If an authorized user interacts with thephone at 740 a, the audio-based authentication may authorize the user at745 and the device will be unlocked with the display on at 740 b.

The audio-based authentication (e.g. generating an inaudible signal,creating a test echoprint and comparing/testing the created testechoprint with a user specific echoprint model) or other applicableautomatic authentication may be performed in a manner unaware to a userof the device. A registered or authorized user of the device may beautomatically authenticated to unlock a device (e.g. turning on adisplay showing a working user interface of an application previouslyused) without a need to perform an alternative authentication operationwhich may require user interface operations (e.g. via a display ormicrophone etc.) before the user can access the previously usedapplication. As a result, a device can activate power saving mode (e.g.display auto off) and/or security protection (e.g. auto locking)mechanisms more often or more effectively without affecting the userexperience.

FIG. 8 depicts an example of typical architecture for a mobileprocessing device phone. The mobile processing device 800 has memory810, a physical connector 820, processor 840, an input/output (I/O)controller 850, a cellular radio channel and data channel 860, and powercontroller 870. Each of these components is connected through the systembus (not shown) of the mobile processing device 800.

Memory 810 includes the mobile processing device's operating system 812,applications 814, and non-volatile storage 830. Memory 810 can be anyvariety of memory storage media types, including non-volatile andvolatile memory. The operating system 812 handles the differentoperations of the mobile processing device 800 and may contain userinterfaces for operations, such as placing and receiving phone calls,text messaging, checking voicemail, and the like. The applications 814can be any assortment of programs, such as a camera application forphotos and/or videos, an address book application, a calendarapplication, a media player, an internet browser, games, an alarmapplication, other third-party applications, and the like. Theapplications can include a signal processor application 110 a adapted toperform the functions of the signal processor 110 when executed by theprocessor 840, a learning processor 120 a adapted to perform thefunctions of the learning processor 110 when executed by the processor840, and a testing processor 140 a adapted to perform the functions ofthe testing processor 110 when executed by the processor 840. Inparticular, the applications may include code 842 adapted to compare atest echoprint with a user-specific echoprint model associated with theuser. The applications may further include code 844 adapted to performan alternative authentication operation to authenticate the user basedon the comparison at 842. The applications may further include code 846adapted to update a device state according to the comparison or thealternative authentication operation performed, with the device statecomprising whether the display 852 is on or off, and whether the deviceis locked or unlocked. The non-volatile storage component 830 in memory810 contains data such as web caches, music, photos, contact data,scheduling data, and other files, as well as echoprints and theuser-specific echoprint model.

The operating system 812 manages the hardware of the mobile processingdevice 800, including hardware such as the display/touchscreen 852,speaker 104, and microphone 102. The operating system 812 also managessoftware (i.e. applications 814) on the mobile processing device 800 forperforming tasks requested by the user and handling incoming data, forexample. This occurs through the operating system's control andallocation memory (i.e. RAM), system tasks, system resources, filessystems, and the like. The processor 840 executes operations for themobile processing device according to this control and allocation.

The power controller 870 of the mobile processing device 800 allocatespower from the mobile processing device's power supply 872 to thecircuitry for different mobile processing device components used tooperate the mobile processing device 800 and its different features.

Additionally, the physical connector 820 can be used to connect themobile processing device 800 to an external power source, such as an ACadapter or powered docking station.

The mobile processing device 800 also contains a cellular radio channeland data channel 860 for receiving and transmitting data, such as phonecalls, text messages, email, webpage data, and the like. Cellular radiocommunication can occur through any of the standard network protocols ofmobile processing device communication (i.e. GSM, PCS, D-AMPS, UMTS, andthe like.). The mobile processing device 800 may also contain additionalcommunication channels 862, such as Wi-Fi, Bluetooth, and the like, forreceiving and transmitting data as well. The mobile processing device800 may have additional functional elements for communication 864, suchas GPS. Each of the described communication mediums is accessed via theantenna 866 on the mobile processing device 800. The communicationmediums for operations of the mobile processing device 800 are notlimited to the mediums described and can include any other communicationmediums known in the art.

It should be recognized that any suitable processing device, mobile orotherwise, may implement the present technology. Hence, although FIG. 8illustrates a mobile device, similar components to those illustrated inFIG. 8 may be provided in a general purpose processing device such as adesktop computer, laptop computer.

Further, in an exemplary, non-limited embodiment, implementations caninclude distributed processing, component/object distributed processing,and parallel processing. Virtual computer system processing can beconstructed to implement one or more of the methods or functionalitiesas described herein, and a processor described herein may be used tosupport a virtual processing environment. Aspects of the presentdisclosure are described herein with reference to flowchartillustrations and/or block diagrams of methods, apparatuses (systems)and computer program products according to embodiments of thedisclosure. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable instruction executionapparatus, create a mechanism for implementing the functions/actsspecified in the flowchart and/or block diagram block or blocks.

In one embodiment, the technology may be implemented by a means foroutputting an inaudible user-specific sound signal at a frequency from aspeaker associated with the processor, such means including by way ofexample a user-specific audio generator 150. The technology may includemeans for receiving reflections of the inaudible sound signal such meansincluding, by way of example, a microphone and signal processor. Thetechnology may include means for generating an echoprint based on thereflections and associating the echoprint with a user which maycomprise, by way of example, a signal processor 110. The technologyincludes means to authenticate a user to provide access to a processingdevice including by way of example, a testing processor.

Some embodiments herein may be implemented in computer-readablenon-transitory media that includes all types of computer readable media,including magnetic storage media, optical storage media, and solid statestorage media and specifically excludes signals.

The terminology used herein is for the purpose of describing particularaspects only and is not intended to be limiting of the disclosure. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The description of the present disclosure has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited to the disclosure in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of thedisclosure. The aspects of the disclosure herein were chosen anddescribed in order to best explain the principles of the disclosure andthe practical application, and to enable others of ordinary skill in theart to understand the disclosure with various modifications as aresuited to the particular use contemplated.

For purposes of this document, each process associated with thedisclosed technology may be performed continuously and by one or morecomputing devices. Each step in a process may be performed by the sameor different computing devices as those used in other steps, and eachstep need not necessarily be performed by a single computing device.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

What is claimed is:
 1. A processing device having audio-basedauthentication, comprising: a display; a microphone; a speaker; an audiosignal generator coupled to the speaker and adapted to output auser-specific inaudible sound signal from the speaker; a signalprocessor coupled to the microphone and adapted to isolate reflectionsof the user-specific inaudible sound signal and create a test echoprintbased on the reflections; and a processor configured to compare the testechoprint with a user-specific echoprint model associated with the user;perform an alternative authentication operation to authenticate the userbased on the comparison; and update a device state according to thecomparison or the alternative authentication operation performed, thedevice state comprising whether the display is on or off, and whetherthe device is locked or unlocked.
 2. The processing device of claim 1further including a machine learning processor adapted to create theuser-specific echoprint during an alternative authentication set-upprocedure.
 3. The processing device of claim 1 wherein the echoprintidentifies features associated with the user, such features including atleast biometric features or behavioral features.
 4. The processingdevice of claim 3 wherein the biometric features include at leastphysical attributes or cardiac attributes.
 5. The processing device ofclaim 1 wherein the machine learning processor is adapted to build theuser-specific echoprint model during a period less than or equal tobeginning when a user enables authentication through when a user selectsto enable audio-based authentication.
 6. The processing device of claim5 wherein the signal processor is adapted to build the user-specificechoprint during a period between entry of a password entry userinterface through a user-interface prompt suggesting the user enableaudio-based authentication.
 7. The processing device of claim 1 furtherincluding a non-volatile storage device adapted to store one or moreechoprints generated by the signal processor as a training data set andthe user-specific echoprint model.
 8. The processing device of claim 1wherein the machine learning processor is adapted to receive userfeedback on authentication recognition and update the user-specificechoprint model based on the feedback.
 9. The processing device of claim1 wherein the audio signal generator and signal processor are adapted tooperate while the display is on and the device is locked, and areadapted to remain in standby when the display is off and the device islocked.
 10. A processor-implemented method of authenticating user accessto a processing device having a display, comprising: generating auser-specific echoprint model using a machine learning process byoutputting an inaudible sound signal at a user-specific frequency from aspeaker associated with the processing device, and receiving reflectionsof the inaudible sound signal at a microphone associated with theprocessing device while the user interacts with an alternativeauthentication user interface; authenticate the user access when thedisplay is on by generating a test echoprint by outputting an inaudibletest sound signal at a user-specific frequency from the speaker, andreceiving reflections of the inaudible test sound signal at themicrophone; comparing the test echoprint with the user-specificechoprint model to determine whether to allow the user to access theprocessing device; performing an alternative authentication method basedon the comparison; and allowing or denying access to the processingdevice based on the testing by updating a device state according to thecomparing or the alternative authentication operation performed, thedevice state comprising whether the display is on or off, and whetherthe device is locked or unlocked.
 11. The processor implemented methodof claim 10 wherein the generating a test echoprint, the receivingreflections of the inaudible test sound signal, and the testing occurwhile the display is on and the device is locked, and do not occur whenthe display is off or the device is locked.
 12. The processorimplemented method of claim 11 wherein the outputting and receiving forgenerating the user-specific echoprint model are repeated a plurality oftimes to create a training dataset for a machine learning process. 13.The processor implemented method of claim 10 wherein the testing outputsa different user-specific inaudible test sound signal upon anauthentication failure, and the testing isolates reflections of thedifferent user-specific inaudible test sound signal and creates a newtest echoprint based on the reflections of the different user-specificinaudible test sound signal.
 14. The processor implemented method ofclaim 10 wherein the generating comprises identifying featuresassociated with the user, such features including at least biometricfeatures or behavioral features.
 15. A non-transitory computer-readablemedium storing computer instructions, that when executed by one or moreprocessors, cause the one or more processors to perform the steps of:generating an echoprint model for a user by: outputting an inaudiblesound signal at a user-specific frequency from a speaker associated withthe processor; receiving reflections of the inaudible sound signal;generating an echoprint based on the reflections and associating theechoprint with a user; repeating the outputting, receiving andgenerating to create training data; training an echoprint model using amachine learning process based on the training data; and authenticatinga user to provide access to a processing device having a display, by,when the display is on and the device is locked: outputting a inaudibletest sound signal at the user-specific frequency from a speakerassociated with the processing device; receiving reflections of theinaudible test sound signal; generating at least one test echoprintbased on the reflections of the inaudible test sound signal; comparingthe echoprint by a user-specific echoprint model to determine whether toallow the user to access the processing device; performing analternative authentication method based on the comparing; and allow ordenying access to the processing device based on the testing and thealternative authentication method by updating a device state accordingto the comparing or the alternative authentication operation performed,the device state comprising whether the display is on or off, andwhether the device is locked or unlocked.
 16. The non-transitorycomputer-readable medium storing computer instructions of claim 15wherein outputting the inaudible test sound includes outputting, for auser, the inaudible test sound signal and a different inaudible testsound and further includes receiving reflections of the differentinaudible test sound and creating an echoprint based on the reflectionsof the different inaudible test sound.
 17. The non-transitorycomputer-readable medium storing computer instructions of claim 15wherein the method further includes providing a user settings interfacefor the alternative authentication method, and the generating andechoprint model includes receiving input from the user in the userinterface while simultaneously performing the outputting, receiving,generating an echoprint, repeating and training.
 18. The non-transitorycomputer-readable medium storing computer instructions of claim 15instructions, that when executed by one or more processors, cause theone or more processors to update the user-specific echoprint model usinga machine learning process based on user feedback of whether the testingwas accurate.
 19. The non-transitory computer-readable medium storingcomputer instructions of claim 15 wherein the echoprint identifiesfeatures associated with the user, such features including at leastbiometric features or behavioral features.
 20. The non-transitorycomputer-readable medium storing computer instructions of claim 19wherein the biometric features include at least physical attributes, orcardiac attributes.